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A NOTE ON IMPLEMENTATIONS OF THE BOOSTING 
ALGORITHM AND HETEROGENEOUS MULTISCALE METHODS 


JOHN MACLEAN* 

Abstract. We present improved convergence results for the Boosting Algorithm (BA), and 
demonstrate that an existing formulation of the Heterogeneous Multiscale Methods (HMM) is ac¬ 
curate to first order only in the macro time step, regardless of the order of the numerical solvers 
employed. These results are obtained by considering the BA and two other formulations of HMM 
as special cases of a general formulation of HMM applied to dissipative stiff ordinary differential 
equations. 
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Introduction. Many problems in the natural sciences are modelled by time-scale 
separated differential equations in which the dynamics of interest takes place on a slow 
time scale. The Heterogeneous Multiscale Methods (HMM) introduced in [5] provide 
the conceptual framework of micro and macro solvers to resolve the behaviour of the 
dynamics of interest without requiring a small integration time step. The micro solver 
performs short, fine-scale computations in the fast variables with micro time steps. 
The output from the micro solver is used to propagate the slow variables for macro 
time steps in the macro solver. 

These methods have been applied to dissipative and to highly oscillatory problems 
[4, 3], as well as multi-scale stochastic problems and partial differential equations 
[21, 1], and to problems exhibiting spatial-scale separation; see [6] for a review. 

There are some differences in the literature on the exact manner in which the HMM 
macro and micro solver interact. In the formulation [6, 21, 22] the micro solver is 
applied during the macro solver whenever a vector field evaluation is required; for 
example, each time step of a fourth order Runge-Kutta macro solver would require 
four applications of the micro solver. We denote this approach by HMMl. 

A different approach to HMM is described in the more recent papers [7, 23, 2], and 
employed in, for instance, [16]. HMM in these papers uses the micro and macro solver 
sequentially; for example, each time step of a fourth order Runge-Kutta macro solver 
would be preceded by an application of the micro solver, and would involve no addi¬ 
tional applications of the micro solver. We denote this approach by HMM2. 

We remark that one cannot distinguish between HMMl and HMM2 in some of the 
literature on HMM, as the two formulations are identical for the most common choice 
of macro solver, the forward Euler method. In this paper we will implement and 
analyse HMMl and HMM2 with a Runge-Kutta macro solver in order to establish 
the important difference between them. 

In practice, the large macro time steps of HMM may require that the fast variables 
be re-initialised in a manner consistent with the slow variables after each application 
of the macro solver. To mitigate this problem, the Boosting Algorithm (BA) was 
proposed in [7] as an extension of the similarly motivated scheme in [18]. The BA 


*School of Mathematics and Statistics, University of Sydney, NSW 2006, Australia 
(J. Maclean@maths. usyd.edu. au) 


1 



2 


A note on implementations of the BA and HMM 


alternates between single applications of the micro and macro solvers, with a smaller 
macro step than HMM. This method was developed in [18, 7] for multi-scale ordinary, 
stochastic and partial differential equations, and extended to highly oscillatory sys¬ 
tems in [15]. In the context of dissipative multi-scale systems, the underlying assump¬ 
tion of the BA is that the slow manifold does not change too much over the smaller 
macro steps, so that the fast variables do not have to be expensively re-initialised or 
re-equilibrated by longer applications of the micro solver as done in HMM. 

A related class of methods called projective methods, proposed in [9], has also been 
applied to time-scale separated ordinary, stochastic and partial differential equations, 
and to bifurcation analysis [12, 10]. Formulations of these methods exist that resem¬ 
ble HMM2 [9, 20, 19] and HMMl [14, 13, 17]. Unlike HMM, in projective methods 
the micro and macro solvers typically propagate both fast and slow variables. This 
allows projective methods to numerically integrate systems in which the slow and fast 
variables are not known; for a discussion of the benefits of this approach, see [22]. 

There is an existing body of work on the convergence of HMM and the BA for or¬ 
dinary differential equations. Of relevance are [4], in which convergence results are 
given for HMM with an arbitrary order macro solver, albeit without proof or explicit 
formulation of the numerical scheme, and [7], in which convergence results are given 
for the BA. 

In this paper we provide rigorous convergence results for HMMl, HMM2 and the BA 
applied to stiff dissipative systems. We confirm for the HMMl formulation the bound 
stated in [4], and show that the HMM2 formulation incurs linear error in the macro 
step regardless of the order of the macro solver. The bound we derive for the BA is 
tighter than the existing bound in [7]; in particular, our bound removes a term that 
implies a restrictive condition should be satisfied by the micro solver. 

The paper is organised as follows. In Section 1 we consider a simple dissipative 
system. In Section 2 we present a general formulation of HMM, with Runge-Kutta 
macro and micro solvers and a variable number of micro steps per application of 
the micro solver. We then consider the BA, HMMl and HMM2 as special cases of 
this general formulation. In Section 3 we establish the convergence of the general 
formulation of HMM to the continuous solution of the slow processes in a multiscale 
system of deterministic ordinary differential equations by a simple extension of the 
convergence proof in [4]. We then apply this bound to the BA, HMMl and HMM2. In 
Section 4 we present results from numerical simulations corroborating our analytical 
findings. We conclude with a discussion in Section 5. 

1. Model. For simplicity of exposition, we restrict our attention to systems with 
one slow variable Xg and one fast variable We consider the deterministic multiscale 
system 



( 1 . 1 ) 

( 1 . 2 ) 


We assume that there exists a slow manifold x = hs{y) = ho{y) + 0(e), towards 
which initial conditions are attracted exponentially fast. On the slow manifold, the 
dynamics slows down and is approximately determined by 
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with reduced dynamics X = + 0{e). The reduced slow vector field is given by 

F{x) = f{x,he{x)) 

= fix,ho{x))+0{e) , (1.4) 

where we identify the zeroth order approximation of the reduced vector field in (1.4) 
because, as we shall see, the HMM and BA approximate f{x, ho{x)) rather than F{x). 

2. Methods. We consider three numerical multiscale methods. Each method 
intermittently resolves the fast dynamics with a micro solver for micro time steps 6t 
to relax the fast variable to the slow manifold, enabling the macro solver to resolve 
the slow dynamics with larger time steps. We select as the micro solver an explicit 
numerical method of order p. The three methods are conceptually described as follows: 
^ 1. BA: The macro solver is a numerical method of order P with macro step 

At ^ St. In this approach the micro solver is applied for a single micro step before 
each time step of the macro solver. As explained in [7], one way to think of the BA 
is as a rescaling of time. In particular, it is equivalent to replacing e in (1.2) with 
Ate/St and then solving the new system of ordinary differential equations by standard 
numerical methods with time step At. Since Ate/St ^ e, the new system is much 
less stiff and the BA presents a large computational advantage over solving (1.I)-(I.2) 
directly. 

2. HMMl: The macro solver is constructed by modifying a numerical method 
of order P to include an application of the micro solver before every vector field 
evaluation of (l.I). The macro step is At 3> At, and the micro solver is applied for 
sufficiently many steps to relax the fast variable. 

3. HMM2: The macro solver is given by a numerical method of order P with 
macro step At At. The micro solver is applied for sufficiently many steps to relax 
the fast variable between each application of the macro solver. 

We will compare the three methods, focussing on a detailed analysis of the BA and the 
differences between HMMl and HMM2. To do so, we consider a general formulation 
of HMM which allows for the construction of the BA, HMMl and HMM2 formulations 
with particular parameter choices. 

Throughout the paper, we use superscripts to denote elements of the macro solver 
and subscripts to denote elements of the micro solver. 

2.1. General formulation of HMM. We describe an HMM scheme in which 
the macro solver is given by a Runge-Kutta method of order P, except that the micro 
solver is applied for a variable number of steps before each vector field evaluation of 
the slow dynamics. Denote by ipm,st the flow map for the micro solver applied to 
the fast dynamics (1.2) with fixed slow variable for m micro steps with time step St. 
Denote by x" and j/" the approximations to the slow and fast variables at macro time 
F. The macro solver is then given by a weighted sum of increments, each of which 
cover a time step At. The increments are generated recursively by 

k^^Hx^,y^) = At/(x" + , (2.1) 

for j = 1, 2,..., P, where we define for j = 1, 2,..., P, m = 1,2,..., Mj , as the 
output of the micro solver 


( 2 . 2 ) 
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with initial condition at j = 1 


and at j > 1 


n,l n 

Vo =y > 


n.j n,l 

Vo = Vm, 


(2.3) 


(2.4) 


The nodes in (2.1), (2.2) are those used in the increments of a Runge-Kutta 
solver of order P; in particular 0 < < 1, and = 0 always so that is 

defined explicitly. 

The macro solver is then given by the weighted sum 




1=1 


(2.5) 


where the weights are appropriate to a Runge-Kutta solver of order P; in particular 
0 < < 1, and = 1- We must estimate on an ad hoc basis; in this 

scheme we choose 


y^^=yX- (2-6) 

The three numerical methods described above can now be considered as special cases 
of the HMM formulation (2.1)-(2.6), and are illustrated in Figure 2.1. 

1. BA: Choose Mi = 1 and all other Mj = 0. As described above, the BA takes 
smaller time steps in the slow variable than the HMM methods; we choose 
the macro step to be At. We choose the number of iterations of the method 
n, so that the time elapsed over an application of the BA is t” = fiAt. The 
BA is illustrated in Figure 2.1a. 

2. HMMl: Fix Mj = M > 0 for all j, so that the micro solver is employed 
before each increment is estimated during the application of the macro solver. 
Choose the macro step At = MAt and number of iterations n = n/M, so 
that the time elapsed over an application of HMMl is t" = nAt = nAt = t” 
in the BA. The formulation HMMl is illustrated in Figure 2.1b. 

3. HMM2: Choose Mi = M as above, and all other Mj = 0, so that the micro 
solver is employed before the first increment is estimated in the macro solver, 
and held fixed during the macro solver. As in HMMl, choose macro step At 
and n iterations so that t” = t”. The formulation HMM2 is illustrated in 
Figure 2.1c. 

3. Error Analysis. We will establish bounds on the error between the slow 
variables a;" given by the general formulation of HMM (2.1)-(2.6) and the true value 
of the reduced slow dynamics A(t"). This provides error bounds for the Boosting 
Algorithm that improve on the bounds obtained in [7], and furthermore establishes 
the distinction between the HMMl and HMM2 formulations. 


We assume that the slow dynamics (1.1) is Lipshitz continuous in the local region in 
which the slow manifold exists, so that there exist constants Lf and Cf satisfying 

\f{xi,yi) - f{x2,y2)\ <Lf{\xi - X2\ + \yi - 2 / 2 !) , 

\f{x,y)\ <Cf . 
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Fig. 2.1: Sketch of the numerical methods for a second order Runge-Kutta macro 
solver. For this scheme = 1 and = 1/2. Initial conditions are (a:", y") at 

the top right; applications of the micro solver are given by crosses while the increments 
are displayed as vectors. To focus on the different ways the increments are defined 
in the three formulations, we omit the final step of the macro solver, which is the 
weighted sum = a;" + + k^'^'>)/2. 


Furthermore we assume that the approximate slow manifold ho(x) is Lipshitz contin¬ 
uous, so that there exists a constant Lh satisfying 

|ho(a:i) - ho(a;2)| < Lh\xi - X2\ ■ 

Combining the previous two assumptions, the reduced dynamics (1.4) is Lipshitz 
continuous, so that there exists a constant Lp satisfying 

\F{Xi)-F{X2)\<Lf\Xi-X2\ , 

with Lp < Lf{l-\-Lh). Under these assumptions, we formulate the following Theorem. 

Theorem 3.1. For the general formulation of HMM deseribed by (2.1)-(2.6), 
with Runge-Kutta macro solver of order P > 1 and micro solver of order p > 1, and 
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for any T > 0, there exists a constant C > 0 such that the error |a:" — X{t’^)\ is 
bounded for t <T by 

|x” - X{r)\ < C (^At^ + |^n,max| ^ ^ 

where p{—5t/e) is the linear amplification factor for the micro solver and is derived 
in the proof on p. 9, and j^^dg-nres the maximal deviation of the fast vari¬ 

able from the slow manifold over all macro steps. Here 0 < p{—6t/e) < 1, and 

gigjiijigg ^/jg exponential attraction of the fast dy¬ 
namics towards the slow manifold over the microsteps. 

In order to apply this result to the formulations BA, HMMl and HMM2, we employ 
the following practical assumptions. 

We assume the parameters 5t, M, At of the HMM and BA solvers satisfy 

St<e< MSt < At < At <1 . 


We assume the micro solver accurately resolves the fast variable compared to the drift 
in the slow manifold over an increment or macro step, with 


\y^-hoix^)\ <LhCfAt 


(3.1) 


in HMMl and HMM2, and 



/io(x°)| < 


LhCfAte 

St 


(3.2) 


in the BA. 


Theorem 3.1 then has the following implications for the three specific implementations 
of HMM described in Section 2: 

Corollary 3.2. For the Boosting Algorithm, 

\x^ - X{n\ < C + e) . 

In particular, the predicted error is |a;" — A(<")| ^ At regardless of the order of the 
macro solver. 

The interpretation of the BA as a rescaling of time, described in Section 2, was used 
in [7] to derive an error bound for the BA. In this interpretation, the BA is thought 
of as a numerical scheme applied to (1.1)-(1.2) with a larger value of e. Consequently 
the error incurred by the BA ’’consists of two parts: The error due to boosting the 
parameter e and the error due to numerical solution of the boosted model” ([7], 
p.5444). This bound can be written as 

|x" - A(t")| < C (Kte/St + At^ + {St/eY^ . 

That is, the error bound presented in [7] predicts the linear scaling of the error with 
At, but also contains an additional term proportional to {St/eY. This term would 
restrict the microstep to <C e and suggest that a higher order micro solver might 
be employed with p > 1, otherwise the BA would incur 0(1) error stemming from the 
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micro solver. Our bound does not have this limitation. In particular, one can choose 
larger 6t to rapidly propagate the fast variable towards the slow manifold, and a hrst 
order micro solver with p = 1 to lessen the computational cost, without increasing 
the predicted error. 

Corollary 3.3. For the Heterogeneous Multiseale Methods in the formulation 
HMMl, 

\x^ -X{t'^)\<c(^At^+ . 


Corollary 3.4. For the Heterogeneous Multiscale Methods in the formulation 
HMM2, 


lx" - X(r)| < C {At^ + At + e) . 


In Corollary 3.3, the error bound may be dominated by any of the three terms. 
However, if e <C At^ and At {—St/e) the error bound for the HMMl 

formulation is identical to the bound for a Runge-Kutta solver applied to the reduced 
system. In Corollary 3.4, employing the practical assumption e < At < 1, the error 
bound is dominated by the linear term in At regardless of the order of the macro 
solver. 

Some descriptions of HMM appear to recommend the HMM2 formulation by describ¬ 
ing the application of the micro and macro solvers separately, as discussed earlier. 
However, Corollaries 3.3 and 3.4 outline the clear advantages of HMMl over HMM2 
for higher-order macro solvers. 


We now prove Theorem 3.1. 

Proof. We follow the procedure established in [5] for bounding the error in HMM 
methods, and in particular we make use of the convergence proof for our system (1.1)- 
(1.2) in [4].^ 

Let us consider the numerical approximation of A(t"), the solution to the reduced 
slow system (1.4) at time t", by a single step of a Runge-Kutta method of order P 
with initial condition X(t"“^). The increments are given by 

((X(t"-i)) =AtF (x(t"-^) -h , (3.3) 

and the solver is then given by the weighted sum 

p 

X" = x(r"i) -f (x(t"-^)) . (3.4) 

1=1 


The Runge-Kutta solver approximates the reduced dynamics to order P -f 1 over a 
single step (see for instance [11]), so that 

X{e) = X{r-^) + AtF (A(r-i)) -H OiAp+^) , (3.5) 


where we write the vector field of the solver as 

p 


F 




1=1 


At 


(3.6) 



A note on implementations of the BA and HMM 


Similarly for the HMM formulation (2.1)-(2.6) we write the vector field of the macro 
solver as 


= fillip , 


i=i 


(3.7) 


(3.8) 


+ C>(At^+^) 


so that 

^n+l ^ Atf{x^) . 

We expand — X{t'^) by substituting (3.5) and (3.8), obtaining 
x" - X(t") =x"-i - A(r-i) + At (/(x"-\ y"-i) - F 

- A(r-i) + At (f(x"-1) - F (A(r-i))) 

+ At (/(x”-\2/'‘-i) -F(x”-i)) +0(At^+i) 

= (x”-^ - A(r-i)) [i + AtF (A(t"-i) + 6»(x”-^ - w(r-i))) 

+ At(/V\2/”"')-A^”-')) +t?(At'^+') , 

where we have applied the Mean Value Theorem with 0 < 0 < 1. Applying absolute 
values and employing the Lipshitz continuity of the reduced dynamics, we obtain 

|x" - A(r)| <(l + LFAt) |x"-i - A(t"-i)| + At /(x"-\y"-i) -F(x"-i) 

+ CAt^+i . 


Iterating with initial condition A(0) = x° and t" < T yields 

pLpT / 


|x" - A(t")| < 


Lf \ 


CAt^ 


max 

l<i<n 


f{x\f)-F{x^) 


(3.9) 


This is the standard bound for HMM methods, established for very general systems 
in [5]. The first term is the error associated with a Runge-Kutta solver applied to the 
reduced system. The second term measures how well the HMM method approximates 
this Runge-Kutta solver, and is labelled e(HMM). 

We now bound |/(x”,j/"') — F(x")|. Substituing (3.6) and (3.7) we obtain 


bU) 






< max 
At 1<3<P 


fc(^)(x",2/”)-if(^)(x") 


(3.10) 


on employing the weighting condition ~ 1- 

We bound |fc(^^(x", 2 /"’) — A'(^^(x")|, substituting (2.1) and (3.3) for the increments 
with 


fc(^')(x”,y”) - F^\x^) =At /(x" -f - F(x” -f 


(3.11) 
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- f{x^ + a^^^K<^^-^\ho{x^ + + 0{e)\ , 

where we used the zeroth order approximation of the slow reduced vector field (1-4). 
Employing the Lipshitz continuity of the slow dynamics and slow manifold yields 

y'li^ - ho{x^ + ) + CAte 


<L 


kU-i) _ i^cO-i) 


- ho{x^ + 


/io(x" + - hoix^ + + CAte 


'ALf{l + Lh)Ati 


,U) 




+ LfAt 2/^ - ho(x" + 


+ CAte . 


(3.12) 


Defining the distance between the fast variable y'!^^ and the approximate slow manifold 
/io(a;” + by 

we write (3.12) with j = 1 as 

fc(i)(x",2/’") - A'(i)(a;") <L/At|d)()^ | + CAte . 

Iterating (3.12), seeded with (3.13), we obtain 
k^^'>{x’^,y^) - K'^^'>{x’^) 

On substitution into (3.10) we obtain 


(3.13) 


<LfAt ^max^ \ + CAte + OiAt"^ max^ |, At^e) . 


f{x^, y”) - F{x’^) < Lf ^max^ | + Ce + 0(At max^ \d^^. |) . 


(3.14) 


We have expressed our bound over the vector fields / and A as a bound over the micro 
steps. The terms Lf maxi<j<p \d^^. \ + Ce measure via (3.11) the mismatch between 
the slow vector held / after an applicaton of the micro solver and the reduced vector 
held F. We now bound \d^^,\. 


Recall that y)()^ is produced by the micro solver, which resolves the fast dynamics 
(1.2) with the slow variable held hxed. At any hxed x, dehne dg = y^ — h^ix), with 
linear dynamics given by 


so that 




(3.15) 


(3.16) 
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By construction, is the output of the micro solver applied to (3.15) with initial 

condition de(0) = — ho{x'^ + We now use this interpretation of 

to establish the rate at which various micro solvers propagate the fast variable to the 
slow manifold. 

It is well known that a single step in a Runge-Kutta solver of order p, when Taylor 
expanded term by term, matches the Taylor series around the previous step to order 
p in (5t. That is, 


d 


n,i 

m+1 





^ dP{d,) 

p\ dtP 


+ 0{StP+^) 


With the linear dynamics of (3.16), the Taylor expansion is unnecessary: every incre¬ 
ment in the Runge-Kutta step from d^^ to can be rewritten in terms of d^^ 

directly. Consequently there is no error term in the above equation, and 


d:ii, = d^^+dtd. 


6t^ •• 

StP dP{dg) 

d™-' ^ 2! 

p\ dtP 


Employing (3.16) we obtain the bound on d^^ as stated in [4], 


d 


n ,7 

m +1 ~ P 



(3.17) 


where 



For the fast variable to converge to the (approximate) slow manifold, we require 
\p{—5t/£)\ < 1, which is assured for any 0 < St < e. Note that that from (3.17) we have 
lim,„_^oo = 0, implying that the fast variable converges to the approximate slow 
manifold ho over the micro solver, rather than converging to the true slow manifold 
hg. Consequently, (3.14) assures that the HMM vector field / converges to the zeroth 
order approximation of the reduced dynamics. 

Substituting (3.17) into (3.14), and (3.14) into (3.9), we obtain the main result of 
Theorem 3.1, that 

Ix’^ - X(t")| < C (^At^ + |^n,max| ^ 

where we defined the maximal deviation of the fast variable from the slow 

manifold over the applications of the macro solver, 

= max . 

0<2<n 

1<3<P 

Equation (3.18) already contains the term that differentiates HMMl from HMM2: in 
HMMl, min(Mj) = M, while in HMM2 (and in BA), min(Mj) = 0. □ 

We now prove Corollary 3.2, 3.3 and 3.4 by bounding each of the three 

formulations. 
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Proof. First fix n and consider |dg’^| over the increments. For j > 1 we employ 
the Lipshitz continuity of the slow dynamics to bound 

Mo’^l =lyMi - 

<|/io(a;”) - - /io(a:”)| 

<LhCfAt + p^^ , (3.19) 

where we have employed < 1, and where we have defined the distance of the fast 
variable from the slow manifold at the beginning of the n-th macro step, 

K| = |y"-/ro(a;”)| . 

For j = 1, Idp’^l = |d”|. Analogously to (3.19) we now bound |ci”|, employing (2.5) 
and (2.6) to obtain 

K+^\=\y^Ml-ho{x-+^)\ 

<|/io(a;”) - /io(a;”^^)| + IVmI - ho{x'^)\ 

<LhCfAt + p^^ . (3.20) 

We note that the bounds for the increments (3.19) and the macro steps (3.20) are the 
same. In each bound, the first term measures the changing value of the slow manifold 
over the drift of the slow variable, and the second term measures the distance of the 
fast variable from the approximate slow manifold after the last macro step. 

We will now use (3.19) and (3.20) to bound for the three formulations HMMl, 

HMM2 and BA. 


Recall that for HMMl and HMM2, Mi = M. For these formulations, we employ the 
practical assumption (3.1) that the micro solver accurately resolves the fast variable 
compared to the drift in the slow manifold over an increment or macro step, which 
we recall as 

(- 7 ) l^"l <LhCfAt . 

Then we obtain to lowest order the bound for HMMl and HMM2, 


|^n,max| < 2LhCfAt 


(3.21) 


For the BA, with Mi = 1, we cannot neglect p^ (—^) M”|; instead we iterate (3.20) 
to obtain 


d”! <p" 



<P 


n 



£\+L,C,At 

d°| + LhCfAt^ ^ ^ {-—) 
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Recalling that h = nM, we can now employ the practical assumption (3.2) that the 
applications of the micro solver dampen any initial distance of the fast variable from 
the slow manifold compared to the drift in the slow manifold over an increment or 
macro step, which we recall as 

^nM f 1^01 ^ LfiCfAt £ 

\ e / ' ' St ’ 

yielding to lowest order the bound 

|^n,max| ^^ ^hCfAtS ^2) 


Substituting (3.22) into (3.18) with min(Mj) = 0 establishes Corollary 3.2; similarly 
(3.21) with min(Mj) = M establishes Corollary 3.3, and (3.21) with min(Mj) = 0 
establishes Corollary 3.4. □ 


4. Numerics. We now illustrate the key predictions of Corollaries 3.2, 3.3, and 
3.4, which we recall here with a second-order Runge-Kutta macro solver, i.e. P = 2, 
and a forward Euler micro solver, i.e. p{—6t/e) = 1 — 6t/e. We present the result 
for each method along with the predicted behaviour of the error |a;" — X(t")| as the 
parameters At, e, St, M are varied. 

1. BA: 


- A(r)| <c 



+ 


Ate 

MSt 


+ e 


where we have used At = At/M so that the BA may be compared to HMMl 
and HMM2. So long as the practical assumptions MSt < At < \, St < e are 
satisfied, the error is always dominated by the second term; in particular the 
error is linear in At and in e. 

2. HMMl: 

|a:”-A(t”)| <c|^At2-HAt(^l-^^ -he 

Depending on the parameters of the method and system, any of the three 
terms could be the dominant error term. 

3. HMM2: 



|a;"-A(t")| ^^(At^-hAt-he) . 

So long as the practical assumption e < At < 1 is satisfied, the error is always 
dominated by the second term; we expect the error to scale linearly with At. 
Employing the practical assumption e < MSt, we see that the second term 
in the bound for HMM2 is larger than the second term in the bound for the 
BA. 

To illustrate these results we present simulations of the Michaelis-Menten system 
employed in [8], 


X, 


■Xe + {Xe -h 0.5)2/, 


(4.1) 
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Ve = 


Xe - {Xe + l)ye 


(4.2) 


with stable fixed point at (0, 0). We comment that the fast dynamics of this system 
is more complicated than our form (1.2), but still converges to the slow manifold 

X X 


he{x) = 


+ £7 


O(e^) 


X + 1 ' ” 2{x + 1)^ 
exponentially quickly, so that Xg converges to the slow reduced system 

X = -X + {X + 0.5)heiX) . 


(4.3) 


(4.4) 


We use initial conditions ^^(0) = 1, ye{0) = he(l). Under these conditions, the fast 
variable in the micro solver still converges to the slow manifold according to 


d 


m+1 


< 



so that Theorem 3.1 and the Corollaries apply. 


We present results for three situations. First we choose parameters so that the error in 
HMMl is dominated by the term proportional to At^. We then demonstrate situations 
in which the error in HMMl is dominated by the term proportional to At (1 — St/e)^, 
and the term proportional to e. 

4.1. HMMl with ideal parameters. We consider the scaling of the error with 
the macro step At for parameters such that HMMl is as accurate as a second-order 
Runge-Kutta solver applied to the reduced system, i.e. |x" — X{t^)\ ^ At^. We 
choose £ = 10“® so that the error is not dominated by the term proportional to e. 
We choose St = 0.2e and M = 30 so that the fast variable converges very close to the 
approximate slow manifold with (1 — St/e)^ = 0.001 <C At. The number of macro 
steps ranges from 10 to 500 to keep T = 5 fixed. Figure 4.1 clearly illustrates the 
identical quadratic scalings of the error in HMMl and the error in a second-order 
Runge-Kutta solver applied to the reduced system, and the linear scaling of the error 
in HMM2 and the BA with At. 

4.2. HMM error dominated by under-resolved fast variable. We now 

consider the scaling of the error with the macro step At when the fast variable does 
not converge close to the slow manifold, so that the error in HMMl is dominated by 
At(l — St/e)^-, that is, the error scaling is linear in At. We choose as before e = 10“® 
and St = 0.2£, now with M = 10 so that (1 — St/e)^ = 0.1 ~ At. The number of 
macro steps again ranges from 10 to 500 to keep t" = 5 fixed. We illustrate the linear 
error scaling with At of all three formulations in Figure 4.2. 

4.3. HMM error dominated by the scale separation. We now consider the 
scaling of the error with e. This term is dominant in HMMl when At^ <C e and the 
fast variable converges well to the approximate slow manifold. In this situation, as 
discussed earlier, HMMl converges to the zeroth order approximation of the reduced 
system f{x,ho{x)) and the error is dominated by the 0(e) difference between the 
approximate reduced system and the true reduced system /(x, he(x)). We choose as 
before St = OAe and M = 30 so that as before the fast variable converge very close to 
the approximate slow manifold. We use At = 0.1 and take n = 50 macro steps. We 
illustrate the linear error scaling with e of the HMMl and BA for e > At^ = 0.01 in 
Figure 4.2. For HMM2, the error term proportional to At always dominates the error 
so long as the practical assumption At ^ e is satisfied, as noted in Corollary 3.4. 
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Fig. 4.1: Plot of the error versus log(At) for fixed time of integration t" = 5 of the 
system (4.1)-(4.2). The crosses labelled ’RK2’ are from second-order Runge-Kutta 
simulations of the reduced system (4.4). The dashed lines are linear regressions with 
slopes of 2.07 for HMMl, 1.06 for HMM2 and 1.00 for the BA. 


5. Discussion. We have extended the proof in [4] to provide convergence results 
for the Heterogeneous Multiscale Methods with a Runge-Kutta macro solver and a 
variable number of micro steps. These results naturally lead to error bounds for 
three distinct numerical methods, all of which are special cases of the general formu¬ 
lation of HMM. In particular, we obtained an improved error bound for the Boosting 
Algorithm over the bound presented in [7]. The bound in [7] contains an additional 
term that suggests a restriction on the micro step to inefficiently small values or on the 
micro solver to high order solvers; the bound we derived does not have this restriction. 

We also obtained an error bound for two distinct formulations of HMM, and used 
these to demonstrate that the micro solver must be employed repeatedly during a sin¬ 
gle step of a Runge-Kutta macro solver, otherwise the error is dominated by a linear 
term in the macro step. If the micro solver is employed before every vector field eval¬ 
uation in the macro solver as done in HMM I, the simulation of the slow variables is 
within 0(e) of the output of a Runge-Kutta solver of arbitrary order applied to the re¬ 
duced system. This provides a word of caution for the conceptual definitions of HMM. 

While the HMMl formulation is accurate, it is significantly more computationally 
expensive than HMM2 or the BA. Consequently, accurate formulations of HMM2 or 
the BA are still of interest. We now discuss some progress that has been made in 
designing such schemes. As we have shown in this paper, any such improvements will 
necessarily improve the estimate of the slow manifold used in the macro solver. 

In [16], a leapfrog coupling is shown to yield second order accuracy in the macro 
step for the HMM2 formulation. In particular, this allows the use of a second order 
implicit macro solver, which is difficult to implement in the HMMl formulation and 







John Maclean 


15 


-2 


-4 


e 

■40 


-6 




I -8 


e 

bO 

o 




-10 


_ -X - - " 


.--X--' 


.-43" 



. - - X- " 


----O' 



.-•X-" 



-O" 


- 12 - 


-141 


i 


-3 


X 


log(Af) 


O HMM1 
X HMM2 
* BA 


Fig. 4.2: Plot of the error versus log(At) for fixed time of integration t" = 5 of the 
system (4.1)-(4.2). The dashed lines are linear regressions with slopes of 1.13 for 
HMMl, 1.05 for HMM2 and 0.98 for the BA. 


advantageous if the macro scale dynamics are also stiff. In the same paper a new 
numerical method is suggested, essentially a formulation of HMM2 with a varying 
number of micro steps before each macro time step. It is shown that this approach is 
advantageous in systems where the scale separation varies. 

In [15] an extension of the BA is developed, called Variable Step size HMM. In this 
scheme the BA macro step varies smoothly from micro to macro values; it is shown 
that this approach enables the BA to achieve second order accuracy in the macro step 
under some conditions on the step size. 

The bound we derived for the BA is partially applicable to VSHMM. In particular 
[15] uses fourth order Runge-Kutta for the micro solver in dissipative experiments 
when our analysis suggests one could gain in accuracy via faster convergence to the 
slow manifold, and save a factor of 4 computationally, by using forward Euler for the 
micro solver. 


Acknowledgements. The author is grateful to Georg Gottwald for many useful 
comments. Any mistakes remain my own. 
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